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(57) ABSTRACT 

This invention provides a method and associated device an 
inverse histogram based pixel mapping step is combined 
with an edge enhancement step such as unsharp masking. In 
such an arrangement the inverse histogram based pixel 
mapping step improves the performance of the unsharp 
masking step, serving to minimize the enhancement of noise 
components while desired signal components are sharpened. 
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DIGITAL IMAGING 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to digital imaging 
technology, and more particularly to contrast and edge 
enhancement in still and moving digital imaging. 

[0002] Digital imaging offers a great many advantages 
over traditional analogue techniques, supporting services 
such as video telephony and multi-media applications. 
Accordingly, digital imaging is currently the subject of 
extensive research. 

[0003] One particular area of digital imaging research is 
that of image enhancement. In digital imaging, there is need 
for image enhancement because images typically suffer from 
various types of degradation, such as lack of sharpness, 
noise, compression artefacts, and poor contrast and bright- 
ness. Mobile imaging products presently under development 
are especially prone to such degradations due to the low 
performance of current digital camera devices and the 
narrow data transmission bandwidth typically available in 
mobile communications networks, which requires the use of 
high compression ratios. In digital camera products, poor 
image quality arises from shortcomings in fundamental 
sensor properties. The lack of contrast typically arises from 
the sensor itself, while the lack of sharpness is the result of 
both low quality optics and low sensor resolution. Low 
resolution sensors are likely to be used in the imaging 
systems of forthcoming first generation mobile telecommu- 
nication terminals with image presentation and manipulation 
functionality. Thus, there is a need for methods to enhance 
image contrast and sharpness, and in current digital image 
enhancement techniques this is commonly carried out by 
concealing artefacts and reconstructing degraded parts of an 
image. 

[0004] Image enhancement principally concerns accentua- 
tion of image features such as edges, boundaries, or contrast. 
This makes image details more visible and makes images 
look subjectively sharper or crisper without appearing 
unnatural, therefore making a graphic display more useful 
for display and analysis. Whilst the enhancement process 
does not increase the inherent information content in the 
data, it does increase the dynamic range of chosen features 
so that they can be more easily detected. 

[0005] Image enhancement techniques can be divided 
roughly into three sub-classes comprising point, spatial, and 
transform operations. 

[0006] Point operations are performed on individual pixels 
and as such are independent of surrounding pixel values. 
Pixel mapping operations are one example of point opera- 
lions. Pixel mapping involves mapping a pixel value to a 
new value with a function, for example: 

[0007] where 

[0008] i,j=pixel*s coordinates 
[0009] y=output pixel 
[0010] x=input pixel 

[0011] and f is a function relating x and y. 



[0012] Spatial operations are performed on groups of 
pixels. Certain image smoothing and filtering functions are 
spatial operations where, for example, a pixel's value may 
be modified by taking an average of surrounding pixel 
values. 

[0013] Transform operations convert image data from 
pixel values into another form, for example spatial fre- 
quency components. Discrete cosine transforms (DCTQ and 
wavelet transforms are examples of transform operations 
commonly used in image processing applications. 

[0014] Examples of enhancement techniques in the sub- 
classes outlined above include techniques such as grey level 
and contrast manipulation, noise reduction, edge crispening 
and sharpening, filtering, interpolation and magnification, 
and pseudo-colouring. 

[0015] In practice, images taken by a digital camera nor- 
mally suffer from smoothness problems irrespective of the 
pixel resolution. This means that there is always likely to be 
a need for image sharpening techniques. Image sharpening 
can be performed by applying various forms of contrast 
enhancement (stretching/shrinking) and edge enhancement 
techniques. Contrast enhancement is a point operation that 
can be used to enhance images taken in poor or non-uniform 
lighting conditions, or acquired with an imaging sensor 
having a small dynamic range. Contrast manipulation also 
tends to alter the visibility of edges, which changes the 
perceived sharpness of the image. Edge enhancement tech- 
niques are typically spatial operations and they enhance 
images that suffer from overall smoothness. A technique 
known as unsharp masking is an example of an edge 
enhancement technique. 

[0016] Referring first to contrast enhancement, a well 
known method for enhancing contrast is histogram-based 
pixel mapping. In this method a pixel-mapping function is 
formed utilising a histogram of the image, or a part of an 
image. The histogram contains the number of pixel occur- 
rences at each level of the input signal range. The mapping 
function is derived from a cumulative histogram so that it 
has high slope corresponding to high values in the histo- 
gram. The cumulative histogram is typically formed using 
an equation having the form: 



*=0 



[0017] where 

[0018] Q-bm i of cumulative histogram 

[0019] H k =bin k of histogram 

[0020] Note that a bin is a pixel value range, that may 
cover a single pixel value or more than one pixel value. 

[0021] In the basic method the mapping function is: 

[0022] The mapping function or the cumulative histogram 
is typically scaled so that the mapping covers substantially 
exactly the available dynamic range, i.e.: 
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[0023] where 

[0024] L=largesl value of the available dynamic 
range 

[0025] The scaling is achieved, for example, using equa- 
tion: 



[0026] The above equations can be found in slighUy 
varying forms in many signal processing textbooks. For 
example, in one alternative approach the first nonzero bin is 
subtracted from the histogram before or during computation 
of the cumulative histogram. If such a subtraction is not 
performed the darkest pixel value of the image is not 
mapped to zero (C o *0). Other modifications also exist. 

[0027] The above equations are usable substantially only 
if there are as many bins as signal levels, for example, 256 
bins are required for a signal with 8 bits/sample. If fewer 
bins are used, each bin contains pixels belonging to a certain 
signal range. The above equations provide mapping function 
values at the borders of bins. Additional values may be 
obtained by interpolation. 

[0028] Examples of a histogram, a cumulative histogram 
and corresponding mapping function are presented in FIG. 
1. 

[0029] Using histogram -based pixel mapping techniques, 
the contrast of an image can be increased by mapping input 
pixel values to output pixel values with a linear function 
having a slope (i.e. derivative) greater than one. Aslope less 
than one decreases contrast, while a slope equal to one 
produces no change in contrast. The mapping function can 
also be non-linear, having a varying slope. In this case 
contrast around some pixel values of the image is increased, 
the contrast in other pixel value ranges may be decreased or 
not altered at all. An example of a non-linear mapping 
function is shown inflG. 2. 

[0030] An acknowledged limitation of histogram-based 
pixel mapping is that it tends to make noise and other 
artefacts disturbingly visible. Because increasing contrast 
increases the visibility of variations in the input signal, if 
there is noise present in the input signal, its visibility is also 
increased. Because of this, histogram -based pixel mapping 
is often considered unsatisfactory for contrast enhancement 
of natural images. 

[0031] Nevertheless, histogram-based pixel mapping is 
the basis for a substantial number of contrast enhancement 
algorithms. So, whilst these algorithms form a mapping 
function based on the image histogram, attempts are made to 
mitigate the unwanted effects of the basic mapping proce- 
dure using modifications and additions to the basic algo- 
rithm. These modifications include, for example, reshaping 
the histogram, reshaping the mapping function, filtering the 
image, the histogram or the mapping function and limiting 
the maximum value of the histogram. 

[0032] Turning now to edge enhancement, specifically in 
the context of unsharp masking, this is a spatial filtering 
technique for improving the visibility of edges and thus 



improving perceived image quality. Unsharp masking is 
commonly implemented in the form of a highpass filter in 
series with an adjustable gain block, as shown in FIG. 3. In 
the illustrated arrangement, a highpass filtered and scaled 
image signal is added to the original image signal as shown 
in FIG. 4a. Thus, the desired improvement is achieved by 
emphasising high frequency components of the image. Spe- 
cifically, the output of the highpass filter represents a second 
order derivative of the original signal, as represented in FIG. 
4b. When the highpass filtered signal is added back to the 
original signal, it creates under and overshoots around edges 
as shown in FIG. 4c. The human visual system itself creates 
similar under and overshoots around sharp edges. This 
phenomenon is known as the Mach band effect. It demon- 
strates that the human visual system perceives edges in a 
non-linear manner, and that perception depends not only on 
luminance but also on image structure. For this reason, the 
human visual system perceives edges with added under and 
over shoot, such as those produced by unsharp masking, as 
sharper edges. 

[0033] Unsharp masking also tends to produce some unde- 
sirable effects. The presence of a highpass filter makes the 
system sensitive to noise and compression artefacts. Thus, 
unsharp masking typically produces perceivable artefacts, 
which are especially visible in uniform areas of an image. 

[0034] Accordingly, it has been recognised that in order to 
improve image quality it would be appropriate to modify an 
input signal of the highpass filter, especially when there is a 
noise component present in an image. One such modifica- 
tion, known in the art, is to use a standard, predefined, or 
fixed pixel mapping function prior to unsharp masking. 
Using a conventional fixed pixel mapping function, it is 
possible to emphasise certain edges, whilst limiting 
unwanted noise enhancement. FIG. 5 depicts an unsharp 
masking scheme that utilises pixel mapping prior to high- 
pass filtering. In the illustrated arrangement, a pixel mapping 
function is applied to the image before unsharp masking in 
order to limit the unwanted enhancement of noise compo- 
nents that occurs as a result of high-pass filtering. 

[0035] FIG. 6 shows two possible pixel mapping function 
curves suitable for use prior to unsharp masking. Curve 1 
suppresses high and low level signals and stretches mid-< 
range pixel values. Accordingly, in the highpass filter, high 
and low level edges are suppressed and mid level edges are 
enhanced. In Curve 2 mid level edges are suppressed and 
high and low level edges are enhanced. In this way, the effect 
of using pixel mapping prior to highpass filtering is to reduce 
noise. 

[0036] The effect of a function such as Curve I, shown in 
FIG. 6, is illustrated in FIG. 7. This shows the effect of 
applying a conventional fixed pixel mapping function to an 
image that contains a single 'edge'. Here the edge is a 
transition from dark to light i.e. from very low intensity pixel 
values to very high intensity pixel values. In other words, the 
image contains just two regions, one of low luminance 
values and one of high luminance values, as shown in FIG. 
la. FIG. lb presents a graph of pixel values created by 
scanning across a row of pixels from left to right. It is 
assumed that the two regions, which should be of essentially 
uniform intensity, exhibit some degree of noise. The noise 
manifests itself as small variations in pixel value (as can be 
seen from the 'ripple' in FIG. lb). By applying the fixed 
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pixel mapping function illustrated in FIG. 7c, the noise in 
the image of FIG. lb can be suppressed effectively. The 
result of the fixed pixel mapping function is illustrated in 
FIG. Id. The way in which the noise is suppressed can be 
understood by examining the pixel mapping function in 
detail. It should be appreciated that the pixel mapping 
function describes the way in which pixel values in the 
original image (i.e. that described by FIG. lb) are converted 
into pixel values in the contrast enhanced (adjusted) image, 
whose pixel value profile is illustrated in FIG. Id. Consid- 
ering first reduction of noise in the low intensity region of 
the image, the relevant part of the pixel mapping function is 
that labelled 'A' in FIG. 7c. This part of the pixel mapping 
function determines how low intensity pixel values in the 
original image are translated into low intensity pixel values 
in the contrast enhanced image. Here the pixel mapping 
function has a gradient less than 1, indicating that variations 
in pixel value will be reduced (or 'compressed') by applying 
the function. The same is true in region *B* of the pixel 
mapping function of FIG. 7c. This part of the curve deter- 
mines the way in which high intensity pixel values in the 
original image are translated into high intensity pixel values 
in the contrast-enhanced image. Thus, noise is effectively 
reduced at both extremes of the range. Referring to how 
mid-range intensity values are converted by the pixel map- 
ping function in this example, the part of the function 
labelled 'C has a gradient greater than 1 and therefore, the 
range of pixel value variation in this region is actually 
enhanced (expanded or stretched), changing the profile of 
the transition between low and high intensity values. How- 
ever, this effect cannot be considered as a true sharpening of 
the transition, such as that provided by unsharp masking. 
Contrast expansion has the effect of enhancing the difference 
between nearly identical mid-range pixel values and inevi- 
tably enhances the effect of noise in the pixel value range 
around the transition. 

[0037] The use of conventional fixed pixel mapping func- 
tions in order to condition an image prior to unsharp 
masking has certain limitations. These will be described 
with reference to FIG. 8, which illustrates the effect of using 
the same pixel mapping function, introduced above, to 
enhance the contrast of a second example image. As in the 
first example, the image considered in FIG. 8 is a very 
simple one, comprising two substantially uniform regions, 
one darker than the other (FIG. 8a). The difference in 
intensity values between the two regions is less than in the 
previous example, although the same kind of noise is present 
(FIG. Sb). The effect of the fixed pixel mapping function is 
illustrated in FIG. Hd. While noise in the darker region is 
suppressed, noise in the lighter region is actually enhanced. 
This effect occurs because the pixel values of the lighter 
region correspond to a region of the pixel mapping function 
that has a gradient greater than one. Thus, small differences 
in pixel value are enhanced, rather than suppressed, causing 
an increase in noise. 

[0038] It is common that images are processed using both 
(fixed or histogram-based) pixel mapping contrast algo- 
rithms and unsharp masking edge enhancement algorithms. 
Referring to FIG. 9, the combination of histogram-based 
pixel mapping and unsharp masking can be implemented by 
placing the histogram-based pixel mapping in the original 
image signal branch before (Option A), after (Option C), or 
in parallel (Option B) with an edge enhancement unit. 
Commonly, image processing applications, such as certain 



commercially available software packages, implement 
options A or C, giving rise to a cascaded structure. However, 
it has been found that the best results are achieved by option 
B, in which a histogram -based pixel mapping unit is placed 
inside the unsharp masking unit, that is in the original image 
signal branch, in parallel with the highpass filter and not in 
cascade with it as in options A and C. In option B, any 
unwanted noise component that is emphasised by contrast 
enhancement, is not highpass filtered and thus not further 
emphasised. In both options A and C the second operation in 
the cascaded structure further emphasises noise artefacts 
introduced by the first operation. Here, the second operation 
refers to the latter operation, i.e. in option A, the second 
operation is unsharp masking, and in option C the second 
operation is histogram-based pixel mapping. Thus, the cas- 
caded approach has the disadvantage that noise is 'doubly' 
enhanced. This can create very annoying noise structures, 
especially if a JPEG or DCT based hybrid codec has been 
used to compress the original source image. For example, 
assuming option A is used, initial contrast enhancement 
causes a certain increase in pixel value noise. This is further 
exaggerated by the following step of edge enhancement 
using e.g. unsharp masking. In the case of option C, because 
the unsharp masking emphasises edges it also heightens 
noise around edges. The amplitude of this noise is then 
further increased by the histogram-based contrast enhance- 
ment. Also, other artefacts caused by the unsharp masking, 
for example ringing effects may become more visible. The 
approach illustrated by option B, on the other hand, avoids 
double noise enhancement. Here, contrast and edge 
enhancement are performed in parallel and the high pass 
filtered image is added to the contrast enhanced image (as 
indicated by the addition symbol in FIG. 9). 

[0039] Nevertheless, the combined image enhancement 
options of FIG. 9 suffer from the problems associated with 
contrast and edge enhancement previously described above. 

SUMMARY OF THE INVENTION 

[0040] Against this background, and in one aspect, the 
present invention resides in a method for image enhance- 
ment of a digital image comprising a set of image pixels, the 
method comprising applying an inverse histogram-based 
mapping function to the image pixels. 

[0041] The inverse histogram -based pixel mapping func- 
tion can be applied alone, or in connection with any of 
options A, B and C described above. In a particularly 
preferred embodiment, inverse histogram-based pixel map- 
ping is applied in connection with option B. 

[0042] Alternatively, an inverse histogram-based pixel 
mapping function is applied in connection with any image 
processing function. Preferably, the inverse histogram -based 
pixel mapping function is applied in connection with an 
image processing function that tends to enhance noise 
visibility. Most preferably, the inverse histogram -based 
pixel mapping function is applied as a pre-processing step 
prior to an image processing function that tends to enhance 
noise visibility. 

[0043] In a preferred form of this aspect of the invention, 
an inverse histogram-based pixel mapping step is combined 
with an edge enhancement step, such as unsharp masking. In 
such an arrangement the inverse histogram -based pixel 
mapping step improves the performance of the unsharp 
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masking step. It also has an advantageous effect on combi- 
nations of contrast enhance mem and unsbarp masking tech- 
niques. 

[0044] As explained earlier, unsbarp masking techniques 
augment the subjective sharpness of an image by adding 
overshoot and undershoot components to edges within an 
image. The effect of inverse histogram-based pixel mapping 
is to change the pixel values according to their frequency of 
occurrence. In embodiments in which the inverse histogram- 
based pixel mapping step is combined with an edge 
enhancement step, such as unsharp masking, the role of the 
inverse histogram based pixel mapping is to act as a pre- 
processor for the highpass filter unit inside the edge 
enhancement unit, that alters pixel values in such a way that 
noise enhancement introduced by subsequent edge enhance- 
ment is reduced. 

[0045] This arrangement serves to minimise the amplifi- 
caton of noise components, while desired signal components 
are enhanced. More specifically, inverse histogram-based 
pixel mapping produces a reduction in contrast at signal 
ranges containing a large number of pixels and an increase 
in contrast elsewhere. Thus, noise is suppressed in e.g. 
uniform areas, where it is subjectively most visible. It has 
been found that the effect is substantially the same when 
inverse histogram -based pixel mapping is applied to: a) a 
large uniform area, and b) an area containing a large number 
of pixels having a small number of different values (e.g. a 
speckled region). 

[0046] According to the first aspect of the invention, there 
is also provided a device, such as a portable radio commu- 
nication device, comprising means for performing said 
image enhancement method. 

[0047] According to the first aspect of the invention, there 
is further provided a software program, comprising machine 
readable instructions for causing a processor unit to perform 
said image enhancement method. The first aspect of the 
invention also extends to a software program product stored 
on a medium and configured to perform said image enhance- 
ment method. 

[0048] According to a second aspect of the invention, 
there is provided a method for image enhancement of a 
digital image comprising a set of image pixels, the method 
comprising applying, in a branch of an edge enhancement 
unit that is parallel to an original image branch, a histogram- 
based pixel mapping function to said set of image pixels 
prior to applying a highpass filter in said branch. 

[0049] As described earlier, use of a fixed pixel mapping 
function in connection with unsharp masking prior to high- 
pass filtering is known, as illustrated in FIG. 5. This has the 
disadvantage that such a function can only be chosen to 
enhance contrast in certain pixel value ranges. Furthermore, 
its effect on noise components in the image varies according 
to the nature of the image, as evidenced by the description 
of FIGS. 7 and 8. Accordingly, the inventors have noted that 
because the function illustrated in FIG. 6 is constructed 
artificially it cannot be altered to adapt to image content and 
is therefore not optimal for all images. 

[0050] In contrast, the inventors have recognised that 
because a histogram-based pixel mapping function is con- 
structed with reference to the image itself, it is adaptive to 



image content and hence some of the problems relating to 
noise enhancement can be reduced. 

[0051] Used in conjunction with edge enhancement, the 
histogram can be adapted such that once a histogram of pixel 
values within an image has been determined, the profile of 
the histogram may be altered in order to change the nature 
of the pixel mapping function created from it 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0052] Referring now to the accompanying drawings, 
FIGS. 1 to 9 are used in conjunction with an account of the 
prior art in which: 

[0053] FIG. 1 shows a histogram and corresponding map- 
ping function; 

[0054] FIG. 2 illustrates a non-linear mapping function; 

[0055] FIG. 3 shows a block diagram of an unsharp 
masking scheme; 

[0056] FIG. 4 illustrates the effect of an unsharp masking 
scheme on an edge of an image signal; 

[0057] FIG. 5 shows an unsharp masking scheme with a 
fixed pixel-mapping unit prior to a highpass filter of the 
unsharp masking scheme; 

[0058] FIG. 6 illustrates two possible curves of a pixel 
mapping function of FIG. 5; 

[0059] FIG. 7 shows the fixed pixel mapping function 
applied to an edge in a first noisy image; 

[0060] FIG. 8 shows a fixed pixel mapping function 
applied to an edge in a second noisy image; 

[0061] FIG. 9 shows possible locations for a contrast 
enhancement unit. 

[0062] The present invention will now be described, by 
way of example only, with reference to the accompanying 
drawings, in which: 

[0063] FIG. 10 illustrates one embodiment of the first 
aspect of the present invention; 

[0064] FIG. U illustrates an inverted histogram con- 
structed according to the embodiment of FIG. 10; 

[0065] FIGS. 12, 13 and 14a-e show alternative arrange- 
ments of the first aspect of the present invention; and 

[0066] FIG. 15 shows an embodiment according to a 
second aspect of the invention. 

DETAILED DESCRIPTION OF THE 
INVENTION 

[0067] The first aspect of the present invention resides in 
the provision of an inverse histogram-based pixel mapping 
technique for enhancing a digital image. As is well known, 
a digital image comprises an array of pixels. In the case of 
a monochrome image, each pixel has a pixel value within a 
certain range (e.g. 0-255), which denotes the pixel's lumi- 
nance. In a colour image, pixel values may be represented in 
a number of different ways. In a commonly used represen- 
tation, referred to as the RGB colour model, each pixel is 
described by three values, one corresponding to the value of 
a Red colour component, another corresponding to the value 
of a Green colour component and the third corresponding to 
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the value of a Blue colour component. Numerous other 
colour models exist, io which alternative representations are 
used. In one such alternative model, known as the YUV 
colour model, image pixels are represented by a luminance 
component and two chrominance or colour difference com- 
ponents, each of which has an associated pixel value. 
Generally, colour models that employ luminance and colour 
difference components provide a more efficient representa- 
tion of a colour image than the RGB model. It is also known 
that the luminance component of such colour models gen- 
erally provides the most information about the perceived 
structure of an image. Therefore, application of the present 
invention may vary according to the colour model used to 
represent a particular image. For example, in an image 
represented using the RGB colour model, the method 
according to the invention may be applied to one, two, or all 
three colour components. On the other hand, if an image is 
represented using the YUV model, it may be more efficient, 
for example, to apply the method according to the invention 
to just the Y component. However, in general, the method 
according to the invention can be applied to all components, 
or any combination (including just one) of the components 
of a particular colour model. Thus, it should be appreciated 
that the choice of colour model is essentially insignificant 
for application of the invention, as the basic principles used 
to construct the inverse histogram-based mapping function 
can be applied to any colour model comprising any number 
of components. 

[0068] Whilst this invention can be used in isolation to 
operate on an image, it has been found that it provides 
particularly beneficial results when used with any image 
processing function that tends to increase noise visibility. 
The approach of the present invention, as well as being a 
method, resides also in a device that includes means for 
applying inverse histogram-based pixel mapping to a digital 
image. In a specific context, the device may be a mobile 
telecommunications terminal, such as a mobile phone, that 
has the conventional radio frequency, hardware and software 
means normally required of a mobile phone and that is 
equipped with image processing capability and/or image 
capturing and display means. Alternatively, the device may 
simply be a computer equipped with digital image process- 
ing means implementing the method according to the inven- 
tion. In the former case, the mobile terminal is able to make 
use of the present invention prior to transmission of a 
captured image, and/or for processing received images for 
display on the mobile terminal. 

[0069] Referring initially to FIG. 10, there is shown one 
embodiment of an inverse histogram-based pixel mapping 
unit that is provided as a pre-conditioning unit for a highpass 
filter unit inside an unsharp masking unit. The inverse 
histogram-based pixel mapping unit is disposed in series 
with the highpass filter that is inside the edge enhancement 
unit and both are in a branch or loop that is parallel to the 
original image signal branch. 

[0070] The inverse histogram -based pixel mapping func- 
tion can be formed, for example, by inverting an image 
histogram and then subtracting the inverted histogram from 
the maximum value of the original histogram. This method 
is illustrated in FIG. U. A standard histogram is first 
obtained from an image and the maximum value of the 
histogram is determined. Then, every value in the histogram 
is subtracted from this maximum value. Next, a cumulative 



histogram is created. Finally, the inverse histogram -based 
pixel mapping function is derived from the cumulative 
histogram. After inversion of the histogram, the mapping 
function is formed in a manner analogous to that used in the 
conventional histogram-based pixel mapping method. It 
should be appreciated that the inverse histogram-based pixel 
mapping function is not an inverse function in a strict 
mathematical sense (i.e. f inv (x)7*r 1 (x)). 

[0071] Thus, in one way the formation of the inverse 
histogram-based mapping function comprises the steps of: 
constructing a histogram of image pixel values, optionally 
processing the histogram, making an inverse histogram from 
the histogram, optionally processing the inverse histogram, 
making a cumulative inverse histogram from the inverse 
histogram, optionally processing the cumulative inverse 
histogram, making an inverse histogram-based pixel map- 
ping function from the inverse histogram, optionally pro- 
cessing the inverse mapping function, and mapping pixel 
values using the inverse histogram-based pixel mapping 
function. 

[0072] The "inversion" can be performed on the cumula- 
tive histogram, or alternatively it can be applied to the 
mapping function. For example, formation of an inverse 
histogram-based pixel mapping function may comprise the 
steps of: constructing a histogram of image pixel values, 
optionally processing the histogram, making a cumulative 
histogram, optionally processing the cumulative histogram, 
making a mapping function from the cumulative histogram, 
optionally processing the mapping function, making an 
inverse mapping function, optionally processing the inverse 
mapping function and, mapping pixel values using the 
inverse histogram-based pixel mapping function. 

[0073] In constructing a histogram it is possible that 
regional histograms could be used, i.e. a histogram repre- 
senting a localised portion of the image. The original his- 
togram may be processed or modified before constructing an 
inverse histogram. Modifications can also be applied to the 
inverted histogram. In addition, it is also possible to make an 
inverse mapping function from a mapping function derived 
from the original histogram. It should also be noted that the 
inverse histogram-based mapping function can be applied to 
all the pixels within an image or to a set of image pixels, for 
example pixels corresponding to a specific region within an 
image. 

[0074] Because the inverse histogram-based pixel map- 
ping function is derived from the inverse histogram via an 
inverse cumulative histogram, and the conventional (non- 
inverted) histogram -based pixel mapping function is derived 
from an image histogram via a cumulative histogram, the 
inverse histogram-based pixel mapping function and con- 
ventional histogram -based pixel mapping function share a 
relationship with one another. Therefore, as explained 
above, there are various ways to construct an inverse his- 
togram-based pixel mapping function from a conventional 
histogram-based pixel mapping function or from a cumula- 
tive histogram. For example, if the conventional histogram- 
based mapping function is differentiated, the derivative is 
inverted (in this context the inversion means subtraction 
from the maximum value) and the result is integrated, the 
obtained mapping function is the inverse histogram -based 
pixel mapping function. 

[0075] In an alternative approach, the inverse histogram- 
based pixel mapping function is produced by subtracting 
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each value of a cumulative histogram (formed in a conven- 
tional manner) from a value larger than the maximum 
histogram value. Furthermore, adding an offset to the maxi- 
mum value allows limitation of the maximum dynamic 
range suppression (no zero slopes). Other modifications can 
also be applied to the inverted histogram prior to formation 
of the inverse histogram-based pixel mapping function. 
What should be noted is the principal shape of the inverted 
histogram compared to the original. Peaks in the original 
histogram are transformed into troughs in the inverse his- 
togram and vice versa. In mapping function terms this means 
that regions of high-gradient are transformed to regions of 
low-gradient and vice versa. 

[0076] Accordingly, the inverse histogram-based pixel 
mapping function is formed so that it has a high slope 
corresponding to low values in the image histogram (i.e. 
pixel value ranges with a small number of occurrences) and 
a low slope corresponding to high values in the image 
histogram (i.e. pixel value ranges with a large number of 
occurrences). The effect of mapping input pixel values to 
output pixel values using this kind of function is that contrast 
is reduced in signal ranges containing a large number of 
pixels and increased elsewhere. This kind of contrast 
manipulation decreases the visibility of small variations and 
noise in the statistically most important signal ranges. For 
example, pixel value ranges that contain large numbers of 
pixels typically correspond to areas of an image having a 
substantially uniform intensity or colour. It is in precisely 
these areas that noise, i.e. small variations in pixel value are 
subjectively most visible. 

[0077] It has been found that by applying a pixel mapping 
function constructed from an inverse histogram of the image 
to the image prior to the edge enhancement unit of an 
unsharp masking block, the problem of amplification of 
noise ordinarily caused by unsharp masking is significantly 
reduced. Noise amplification is particularly disturbing when 
a JPEG or DCT based hybrid codec has been used to 
compress the source image, and the present invention has 
been found to be particularly useful when applied to such 
images. 

[0078] Because inverse histogram-based pixel mapping 
reduces contrast and noise visibility in statistically important 
signal ranges, noise enhancement produced by unsharp 
masking is also reduced at these signal ranges. This has an 
advantageous effect, for instance, when the image contains 
a relatively wide and smooth area. Such an extensive smooth 
area gives rise to a concentration of high values in the image 
histogram. Inverse histogram-based pixel mapping has the 
effect of reducing noise amplification due to subsequent 
unsharp masking, particularly in such a large and smooth 
area. Elsewhere the unsharp masking sharpens edges with- 
out restrictions. Thus, while edges are suppressed in one part 
of the dynamic range e.g. for pixel values representing a 
large smooth area, they are enhanced elsewhere in the 
dynamic range. Therefore "real edges" between regions are 
enhanced without restrictions. 

[0079] As previously explained, image processing opera- 
tions often include both unsharp masking and a histogram 
based pixel-mapping algorithm for contrast enhancement. In 
this case, it has been found that application of the method 
according to the present invention significandy improves the 
resulting picture quality. Inverse histogram-based pixel 



mapping reduces noise amplification in the areas of the 
image that are emphasised by the contrast enhancement 
algorithm. Due to this feature, both algorithms do not 
enhance the noise. 

[0080] FIG. 12 shows an example of an implementation 
of the present invention in which inverse histogram -based 
pixel mapping is used in conjunction with conventional 
histogram-based pixel mapping contrast enhancement and 
unsharp masking. A conventional histogram -based pixel 
mapping contrast enhancement unit is placed in the original 
image branch of the unsharp masking unit, and a combina- 
tion of an inverse histogram-based pixel-mapping unit and 
an unsharp masking unit is placed in a branch that is parallel 
to the original image branch. The output of the parallel 
branch is added to the output of the histogram-based pixel 
mapping contrast enhancement unit. 

[0081] FIG. 13 shows a modified arrangement in which an 
additional (low-pass) filtering block added in the original 
image branch (into the upper path), after the contrast 
enhancement operation. The purpose of the filtering block is 
to reduce the noise introduced by conventional histogram- 
based contrast enhancement. 

[0082] A second aspect of the invention is depicted in 
FIG. 15, in which there is provided a histogram-based pixel 
mapping unit alongside and prior to an unsharp masking unit 
in a branch of an image enhancement arrangement, the 
branch being parallel to the original image branch. The 
output of the parallel branch is added to the original image. 
With this arrangement because the mapping function adapts 
to the image content, noise enhancement has been found to 
be reduced. 

[0083] The present invention may also be applied for 
enhancement of digital video sequences, in which case it is 
advantageous to add temporal processing to the histogram. 
A video signal consists of a sequence of digital images. Each 
image can be enhanced individually, but the same histogram 
cannot be used for two images that have different content. 
This means that histograms are preferably gathered for each 
image. However, processing images in the video sequence 
individually produces disturbing flicker, because even quite 
unnoticeable visual differences can cause two images to 
have substantially different histograms. Therefore, temporal 
processing, for example low pass filtering, of the histograms 
is needed. Typically this kind of processing is adaptive to 
sudden large changes in image characteristics, for example 
due to scene-cuts. 

[0084] The present invention may be embodied in other 
specific forms without departing from its essential attributes. 
The inverse histogram-based pixel mapping function of the 
present invention has been described above in the context of 
a pre-processor for an edge enhancement function. How- 
ever, the present invention may be used on its own or in a 
variety of different ways. FIG. 14 illustrates a number of 
different applications of an inverse histogram-based map- 
ping function unit. FIG. 14a shows the inverse histogram- 
based mapping unit applied on its own to an image. FIG. 
14b shows the inverse histogram-based mapping unit as a 
post-processing unit for an edge enhancement unit. FIG. 14c 
shows the inverse histogram based mapping unit as a 
pre-processing unit for an edge enhancement unit. FIG. 14d 
shows an inverse histogram -based mapping unit in parallel 
with an edge enhancement unit. FIG. 14e shows two inverse 
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histogram-based mapping units, one in an original image 
branch and used as a pre-processing unit for a low pass filter, 
and the other one in a parallel branch, as a pre-processing 
unit for an edge enhancement unit. Other inverse histogram- 
based mapping arrangements are possible, in which the 
method comprises applying the image's statistical data- 
dependent pixel mapping function to the image pixels in 
such a way that image contrast is reduced in signal ranges 
containing a large number of pixels and increased in other 
signal ranges. 

[0085] Accordingly reference should be made to the 
appended claims and other general statements herein rather 
than to the foregoing specific description as indicating the 
scope of invention. 

[0086] Furthermore, each feature disclosed in this speci- 
fication (which term includes the claims) and/or shown in 
the drawings may be incorporated in the invention indepen- 
dently of other disclosed and/or illustrated features. In this 
regard, the invention includes any novel features or combi- 
nation of features disclosed herein either explicitly or any 
generalisation thereof irrespective of whether or not it 
relates to the claimed invention or mitigates any or all of the 
problems addressed. 

[0087] The appended abstract as filed herewith is included 
in the specification by reference. 

What is claimed is: 

1. A method for image enhancement of a digital image 
comprising a set of image pixels, the method comprising 
applying an inverse histogram-based mapping function to 
the image pixels. 

2. A method for image enhancement of a digital image 
comprising image pixels, the method comprising applying 
an inverse histogram-based mapping function to a set of said 
image pixels. 

3. A method according to claim 2, wherein the set of said 
image pixels comprises all image pixels of the digital image. 

4. A method according to claim 2, wherein the set of said 
image pixels comprises a part of all image pixels of the 
digital image. 

5. A method according to claim 2, wherein formation of 
the inverse histogram-based mapping function comprises: 

constructing a histogram of pixel values from at least part 
of said image pixels, 

developing an inverse histogram from said histogram, 

forming a cumulative inverse histogram from said inverse 
histogram, 

deriving an inverse mapping function from said cumula- 
tive inverse histogram. 

6. A method according to claim 5, wherein the step of 
constructing a histogram of pixel values comprises the steps 
of: 

defining a set of pixel value ranges; 

for each pixel value range, counting the number of image 
pixels from said at least part of said image pixels that 
have values within that range. 

7. A method according to claim 6, wherein said step of 
developing an inverse histogram from said histogram com- 
prises the steps of: 



identifying a pixel value range having a maximum num- 
ber of image pixels; 

for each pixel value range, subtracting the number of 
image pixels within that range from said maximum 
number, thus modifying the number of image pixels 
within each pixel value range. 

8. A method according to claim 6, wherein said step of 
developing an inverse histogram from said histogram com- 
prises the steps of: 

defining a reference value; 

for each pixel value range, subtracting the number of 
image pixels within that range from said reference 
value, thus modifying the number of image pixels 
within each pixel value range. 

9. A method according to claim 6, wherein said step of 
developing an inverse histogram from said histogram com- 
prises the steps of: 

identifying a pixel value range having a maximum num- 
ber of image pixels; 

defining a reference value greater than said maximum 
number; 

for each pixel value range, subtracting the number of 
image pixels within that range from said reference 
value, thus modifying the number of image pixels 
within each pixel value range. 

10. A method according to any of claims 7, 8 or 9, wherein 
said step of forming a cumulative inverse histogram com- 
prises the step of: 

for each pixel value range, adding to the number of image 
pixels within that range, modified during said step of 
developing an inverse histogram, the number of image 
pixels modified during said step of developing an 
inverse histogram in all pixel value ranges comprising 
smaller pixel values than that range. 

11. A method according to claim 10, wherein said inverse 
mapping function is derived from said cumulative inverse 
histogram using the number of image pixels within each 
pixel value range of said cumulative inverse histogram. 

12. A method according to claim 11, wherein said inverse 
mapping function is derived from said cumulative inverse 
histogram by interpolating between pixel value ranges of 
said cumulative inverse histogram. 

13. A method according to any preceding claim, wherein 
said inverse mapping function is scaled to a predetermined 
maximum value. 

14. A method according to any of claims 5 to 13, wherein 
said histogram of pixel values is processed to form a 
modified histogram before said step of developing an 
inverse histogram. 

15. A method according to any of claims 5 to 14, wherein 
said inverse histogram is processed to form a modified 
inverse histogram before said step of forming a cumulative 
inverse histogram. 

16. A method according to any of claims 5 to 15, wherein 
said cumulative inverse histogram is processed to form a 
modified cumulative inverse histogram before said step of 
deriving an inverse mapping function. 

17. A method according to claim 2, wherein formation of 
the inverse histogram-based mapping function comprises: 



07/02/2004, EAST Version: 1.4.1 



US 2001/0046320 Al 



8 



Nov. 29, 2001 



constructing a histogram of pixel values from at least part 
of said image pixel values, 

forming a cumulative histogram from said histogram, 

deriving a mapping function from the cumulative histo- 
gram, 

forming an inverse mapping function from said mapping 
function. 

18. A method according to claim 17, wherein said step of 
forming an inverse mapping function from said mapping 
function comprises the steps of: 

differentiating said mapping function to form a differen- 
tiated mapping function; 

finding the maximum value of said differentiated mapping 
function; 

subtracting said maximum value from said differentiated 
mapping function and integrating the result. 

19. A method according to claim 17, wherein the step of 
constructing a histogram of pixel values comprises the steps 
of: 

defining a set of pixel value ranges; 

for each pixel value range, counting the number of image 
pixels from said at least part of said image pixels that 
have values within that range. 

20. Amethod according to claims 19, wherein said step of 
forming a cumulative histogram comprises the step of: 

for each pixel value range, adding to the number of image 
pixels within that range the number of image pixels in 
all pixel value ranges comprising smaller pixel values 
than that range. 

21. A method according to claim 20, wherein said map- 
ping function is derived from said cumulative histogram 
using the number of image pixels within each pixel value 
range of said cumulative histogram. 

22. A method according to claim 21, wherein said map- 
ping function is derived from said cumulative histogram by 
interpolating between pixel value ranges of said cumulative 
histogram. 

23. A method according to any of claims 17 to 22, wherein 
said histogram of pixel values is processed to form a 
modified histogram before said step of developing a cumu- 
lative histogram. 

24. Amethod according to any of claims 17 to 23, wherein 
said cumulative histogram is processed to form a modified 
cumulative histogram before said step of deriving a mapping 
function. 

25. A method according to any of claims 17 to 24, wherein 
said mapping function is processed to form a modified 
mapping function before said step of forming an inverse 
mapping function. 

26. A method according to any preceding claim, wherein 
said inverse mapping function is modified to form a modi- 
fied inverse mapping function and said modified inverse 
mapping function is applied to the set of said image pixels. 

27. A method according to any of claims 5 to 26, wherein 
said histogram is constructed from all image pixels of said 
digital image. 

28. A method according to any of claims 5 to 26, wherein 
said histogram is constructed from a part of all image pixels 
of said digital image. 



29. Amethod according to any preceding claim, wherein 
said inverse histogram-based mapping function is applied to 
the set of said image pixels prior to or after applying another 
image processing function. 

30. A method according to claim 29, wherein said other 
image processing function has the effect of increasing pixel 
value variation. 

31. Amethod according to any preceding claim, wherein 
the inverse histogram-based mapping function is applied to 
the set of said image pixels prior to or after applying an edge 
enhancement function. 

32. A method according to any preceding claim, wherein 
the inverse histogram -based pixel mapping function is 
applied in an original signal branch and an edge enhance- 
ment unit is applied in a branch parallel to the original signal 
branch. 

33. Amethod according to any preceding claim, wherein 
the inverse histogram-based mapping function is applied in 
an original signal branch and in a branch parallel to the 
original signal branch having an edge enhancement unit. 

34. Image processing means for enhancing a digital image 
comprising a set of image pixels, the processing means 
comprising means for applying an inverse histogram -based 
mapping function to the image pixels. 

35. An image processor for enhancing a digital image 
comprising image pixels, wherein the image processor com- 
prises means for applying an inverse histogram-based map- 
ping function to a set of said image pixels. 

36. An image processor according to claim 35, wherein 
the image processor comprises: 

means for constructing a histogram of pixel values from 
at least part of said image pixels, 

means for developing an inverse histogram from said 
histogram, 

means for forming a cumulative inverse histogram from 
said inverse histogram, 

means for deriving an inverse mapping function from said 
cumulative inverse histogram. 

37. An image processor according to claim 36, wherein 
the image processor comprises means for forming a modi- 
fied histogram from said histogram of pixel values. 

38. An image processor according to claim 36 or 37, 
wherein the image processor comprises means for forming 
a modified inverse histogram from said inverse histogram. 

39. An image processor according to any of claims 36 to 
38, wherein the image processor comprises means for form- 
ing a modified cumulative inverse histogram from said 
cumulative inverse histogram. 

40. An image processor according to claim 35, wherein 
the image processor comprises: 

means for constructing a histogram of pixel values from 
at least part of said image pixel values, 

means for forming a cumulative histogram from said 
histogram, 

means for deriving a mapping function from the cumu- 
lative histogram, 

means for forming an inverse mapping function from said 
mapping function. 
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41. An image processor according to claim 40, wherein 
the image processor comprises means for forming a modi- 
fied histogram from said histogram of pixel values. 

42. An image processor according to claim 40 or 41, 
wherein the image processor comprises means for forming 
a modified cumulative histogram from said cumulative 
histogram. 

43. An image processor according to any of claims 40 to 

42, wherein the image processor comprises means for form- 
ing a modified mapping function from said mapping func- 
tion. 

44. An image processor according to any of claims 35 to 

43, wherein the image processor comprises means for form- 
ing a modified inverse mapping function from said inverse 
mapping function. 

45. A portable radio communication device including 
image processing means for enhancing a digital image 
comprising a set of image pixels, the processing means 
comprising means for applying an inverse histogram map- 
ping function to the image pixels. 

46. A computer program product having a computer 
usable medium having computer readable program code 
embodied therein for image enhancement of a digital image 



comprising a set of image pixels, the code comprising 
applying an inverse histogram-based mapping function to 
the image pixels. 

47. A method for image enhancement of a digital image 
comprising a set of image pixels, the method comprising 
applying, in a branch of an edge enhancement unit that is 
parallel to an original image branch, a histogram-based pixel 
mapping function to said set of image pixels prior to 
applying a highpass filter in said branch. 

48. A method for image enhancement of a digital image 
comprising image pixels, the method comprising applying, 
in a signal branch parallel to an original image branch, a 
histogram-based pixel mapping function to a set of said 
image pixels prior to applying an image processing function 
that has the effect of increasing pixel value variation. 

49. A method for image enhancement of a digital image 
comprising image pixels, the method comprising applying, 
a histogram -based pixel mapping function to a set of said 
image pixels prior to applying an image processing function 
that has the effect of increasing pixel value variation. 

* * * * * 
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